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Abstract 

The instantaneous relative entropy (IRE) and the corresponding instantaneous Fisher 
information matrix (IFIM) for transient stochastic processes are presented in this paper. 
These novel tools for sensitivity analysis of stochastic models serve as an extension of the 
well known relative entropy rate (RER) and the corresponding Fisher information matrix 
(FIM) that apply to stationary processes. Three cases are studied here, discrete-time Markov 
chains, continuous-time Markov chains and stochastic differential equations. A biological 
reaction network is presented as a demonstration numerical example. 


1 Introduction 

Sensitivity analysis, for a general mathematical model, is defined to be the quantification 
of system response to parameter perturbations. Questions on the robustness, (structural) 
identifiability, experimental design, uncertainty quantification, estimation and control can 
be addressed through sensitivity analysis [T]. Moreover, it is a necessary analysis tool for 
the study of kinetic models such as chemical and biochemical reaction networks (TJ [5] . The 
mathematical models considered in this paper are models that describe phenomena exhibit¬ 
ing stochasticity: stochastic differential equations (SDE), discrete-time and continuous-time 
Markov chains (DTMC and CTMC). Some of the mathematical tools for the sensitivity anal¬ 
ysis of such systems include log-likelihood methods and Girsanov transformations IS1I1I5], 
polynomial chaos [5], finite difference methods [TIIH] and pathwise sensitivity methods [S]. 
Closely related to the log-likelihood methods are various linear-response-based approaches, 
for instance in the context of chemical kinetics |10| . as well as in recent mathematical work 
for linear response in non-equilibrium systems [min]. 

In [13j . the authors propose a new methodology for the pathwise sensitivity analysis of 
complex stochastic stationary dynamics based on the Relative Entropy (RE) between path 
distributions and Relative Entropy Rate (RER). This two quantities provide a measure of the 
sensitivity of the entire time-series distribution. The space of all such time-series is referred in 
probability theory as the “path space”. RER measures the loss of information per unit time 
in path space after an arbitrary perturbation of parameter combinations. Moreover, RER 
and the corresponding Fisher Information Matrix (FIM) become computationally feasible 
in certain cases as they admit explicit formulas. In fact, it is been showed in m that 
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the proposed pathwise sensitivity analysis has the following properties: (a) it is rigorously 
valid for the sensitivity of long-time, stationary dynamics, (b) it is a gradient-free sensitivity 
analysis method suitable for high-dimensional parameter spaces, (c) the computation of RER 
and FIM does not require the explicit knowledge of the equilibrium probability distribution, 
relying only on information for local dynamics, i.e. it is suitable for non-equilibrium systems. 

In this paper we extend the RE and FIM tools, developed in [13] for stationary processes, 
to transient and non-stationary processes. The extension is based on the notion of instan¬ 
taneous RE (IRE) and instantaneous FIM (IFIM), see for example equations (10) and (12) 
in text. These sensitivity tools provide an instantaneous measure for the sensitivity of a 
system arising from information theoretic tools. Moreover, both IRE and IFIM, which are 
independent of observable functions, can be used as upper bounds for observable depended 
sensitivities, see for example the discussion in Conclusions and Hiiii]. 

The rest of the paper is organized as follows: In Section 2 a review of the RE and 
the FIM is given. In Sections 3,4 and 5 the new quantities IRE and IFIM are presented 
for discrete-time Markov chains, continuous-time Markov chains and stochastic differential 
equations, respectively. In Section 6, a numerical example for a biological reaction network is 
utilized and IRE and IFIM are computed in the course of a stochastic simulation and various 
observations are discussed. Finally, in Section 7, concluding remarks and connections with 
existing works are discussed. 


2 Time-dependent sensitivity analysis 

2.1 Decomposition of the pathwise relative entropy 

The relative entropy (or Kullback-Leibler divergence) of a probability measure P with 
respect to (w.r.t.) another probability measure P is defined as [Ml HD 


(P I P) 


/logfdP, P«P 
oo , otherwise 


( 1 ) 


where ^ is a function known as the Radon-Nikodym derivative which is well-defined when 
P is absolutely continuous w.r.t. P (denoted as P P) while the integration is performed 
w.r.t. the probability measure P. Relative entropy has been utilized in a diverge range of 
scientific fields from statistical mechanics [18] to telecommunications m and finance [19] 
and possesses the following fundamental properties: 

(i) P(P|P)>0, 

(ii) 77. (P I P) = 0 if and only if P = P P-almost everywhere, and, 

(iii) 77 (P I P) < oo if and only if P and P are absolutely continuous w.r.t. each other. 

These properties allow us to view relative entropy as a “distance” (more precisely a diver¬ 
gence) between two probability measures capturing the relative importance of uncertainties 
[20] . From an information theory perspective, relative entropy measures the loss/change of 
information when P is considered instead of P m 

Let a stochastic process -either discrete-time or continuous-time- be denoted by Xt and 
let the path space X be the set of all trajectories {Xt}f^Q. We denote by Qo-.t the path space 
distribution, i.e., the probability to see a particular element of path space, X. Denote by 
(5 o:T the path space distribution of another process, Xi. The pathwise relative entropy of the 
distribution Qo:T w.r.t. the distribution QoiT assuming that they are absolutely continuous 
w.r.t. each other is written using ([^ as 

77 (QoJ I Qo:t) = [ log ■ (2) 

J dQo-.T 
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A key property of pathwise relative entropy is that it is an increasing function of time 
which is the analog of the second thermodynamic law in statistical physics To this end, 
for various important Markov processes, it can be shown, exploiting the Markov property, 
that the pathwise relative entropy can be written as an averaged quantity as 


{Qo:T I Qo:t) 


TZ{i'\h') + 'H (Qi I Qi) , if time is discrete 

TZ{i'\i') + fj' % {Qt I Qt) dt , if time is continuous , 


where TZ{i'\v) is the relative entropy of the initial distribution v w.r.t. the perturbed 
initial distribution D, while H (Q. | Q.) denotes the instantaneous relative entropy (for explicit 
formulas we refer to equation ([l0|, (16) and (24)). We named the quantity % {Q. IQ.) 


instantaneous relative entropy because it is also non-negative but more importantly is the 
time-derivative of the pathwise relative entropy and inherits the properties of the relative 
entropy. Notice that the notation for instantaneous relative entropy is somewhat confusing 
but it will become clear when specific examples will be presented in the following sections. 

Moreover the relative entropy rate is defined for a broad class of stochastic processes 
including Markov and semi-Markov processes as [21] as the limit 


'H{Q\Q) = lim —TZ (Qo:T I Qo:t) 

T —>00 1 


(4) 


Relative entropy rate is closely related to the instantaneous relative entropy in two funda¬ 
mental ways, Firstly it is the limit of instantaneous relative entropy as time goes to infinity, 
whenever this limit is defined and second it holds at the stationary regime that 

n ( q . \Q,)=n{Q\ Q) ( 5 ) 


for the discrete-time case with i = 1, ...,T and similarly for the continuous-time case. 


2.2 Sensitivity analysis and Fisher information matrix 

In [13] ■ authors proposed the relative entropy between path distributions to perform 
sensitivity analysis arguing that pathwise relative entropy takes into account not only the 
equilibrium properties but also the complex dynamics of a stochastic process. Specifically, 
denote by Qq.j^ the path space distribution of the process Xt, parametrized by the parameter 
vector 9 G K*-. Consider also a perturbation vector, e G , and denote by the 

path space distribution of the perturbed process Xt- The pathwise relative entropy of the 
distribution Qq.^ w.r.t. the distribution Qq^ assuming that they are absolutely continuous 
w.r.t. each other is written from ([^ as 

(Qo:T I QlP) = f log • (6) 

An attractive approach to sensitivity analysis that is rigorously based on relative entropy 
calculations is the Fisher Information Matrix (FIM). Indeed, assuming smoothness in the 
parameter vector, it is straightforward to obtain the following expansion for ® [HIES], 

{QI:t I = \e^T{Ql..Ty + 0(|e|'), (7) 

where the K x K pathwise FIM l((3g.y) is defined as the Hessian of the pathwise relative 
entropy. As ([^ readily suggests, relative entropy is locally a quadratic function of the 
parameter vector 9. Indeed, the pathwise RE for any perturbation can be recovered up 
to third-order utilizing only the pathwise FIM. Moreover, similar expansions hold for the 
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instantaneous relative entropy and the relative entropy rate. The decomposition of the 
pathwise FIM reads 


^(Qo:t) 


) if time is discrete 

T'H{Qt)dt , if time is continuous 


( 8 ) 


where is the FIM of the initial distribution while X-h (Q^) denotes the instantaneous 

FIM. In the following sections concrete examples of stochastic Markov processes are presented 
whose pathwise relative entropy and the associated pathwise FIM is provided. 


3 Discrete-time Markov chains 


This section presents explicit formulas of the various relative entropy quantities defined 
in the previous section as well as the associated FIMs for the case of discrete-time Markov 
chains. The analysis of the DTMC case serves (a) as a more intnitive and manageable 
example of stochastic processes and (b) as a intermediate step to handle the continuous-time 
Markov chain case. 

Next, let be a discrete-time time-homogeneous Markov chain with separable 

state space E. The transition probability kernel of the Markov chain denoted by P^{x, dx') 
depends on the parameter vector 9 € Assume that the transition kernel is absolutely 

continuous with respect to the Lebesgue measure and the transition probability density 
function p^{x,x') is always positive for all x,x' € E and for all 9 G Exploiting the 

Markov property, the path space probability density Qq.j^ for the path {xijJLg at the time 
horizon 0,1,..., T starting from the initial distribution (x)dx is given by 

QI,t{xq, ■■■,xt) = i^\xo)p^{xo,Xi).. .p^{xt-i,xt) ■ 

We consider a perturbation vector e G and the Markov chain {xi}igz+ with transition 
probability density function, p^~^'^{x,x'), initial density, iy^^^{x), as well as path distribution 
Qq'^. The prodnct representation of the path distributions results in an additive repre¬ 
sentation of the relative entropy of the path distribution Qq.j’ w.r.t. the perturbed path 
distribution Qq'^. Let i^f{x) denote the probability density function of the Markov chain 
at time instant i given that the initial distribution is , whose formula is provided by the 
Chapman-Kolmogorov equation. 


/ •••/ ’^'^{xo)p'^{xo,xi).. .p^{xt-i,x)dxo ■ ■-dxi-i . 

J E J E 

The following theorem presents the decomposition of the pathwise relative entropy. 

Theorem 3.1. (a) The pathwise relative entropy for the above-defined discrete-time Markov 
chain is decomposed as 


n {Qir I QoP) ^"+0 + E ^ I Q' 




where the instantaneous relative entropy equals to 


n {Qt I Q 


I 


= E„ 


/(a;,a;')log 


^je 


p\x,x') 
p^+'^{x, x') 


dx' 


(9) 


( 10 ) 


(b) Under smoothness assumption on the transition probability function for the parameter 9, 
the pathwise FIM is also decomposed as 


x{qU) =x{v 0) + Y,MQt) , 


( 11 ) 
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where the instantaneous pathwise FIM is given by 

P®(a;,x')Velog/(x,a;')Velog/(a;,a;')^da;'] (12) 

‘ JE 

Proof, (a) The proof of this part of the theorem can be found in [T71 Ch. 2] under the title 
“Chain rule for relative entropy” but for the shake of completeness we present it here. The 
Radon-Nikodym derivative of the unperturbed path distribution w.r.t. the perturbed path 
distribution takes the form 

/'r™ ^T \ _ 


{{x^}J=o) = 


^ ^«+^(xo)nfo 

which is well-defined since the transition probabilities are always positive. Then, 

^ {Qo:T I Qcet) 

f f s, ^TT e, M ’^Axo)Uf=iPAxi-i,x,) ^ ^ 

= ■■■ ^ (a;o)llp {xj-i,Xj)log -—- -dxo...dxT 

Je Je E^+'^{xo)l{i^^P^+Axi-l,Xi) 

= j ■■■ j v^xo) Y[pAxj-i,Xj)\og JeJ'^°J^^ d.xo ...dxr 


E 


E 


{xo)W_p\xj-i,Xj) log 


i=i 


P {.Xi— 1 , 

P^+'^{Xi-i,Xi) 


dxo ...dxT 




2=1 


where TZ {y^ \ = E^e log A[x) is the relative entropy of the unperturbed initial distribu¬ 

tion w.r.t. the perturbed one, while the instantaneous relative entropy (of the time-varying 
pathwise relative entropy) is 

n (<3? 10“+-) = Ku [ !/(=-:• =^’1 

(b) The proof of this part of the theorem is similar to the proof of the pathwise FIM for 
the relative entropy rate in m- We present it with minor but necessary adaptations. Let 
Sp{x,x') = p^^'^{x,x') — p^{x,x'), then the instantaneous relative entropy "H (Qf | at 

the Tth time instant is written as 

{Qi I = - [ f ^!-iix)pAx, x') log ("l + ) dxdx' 

Je JE \ P ) / 


E Je 




E^_^{x)Sp{x,x') --E,{x) 

2 p^{x,x') 


oiMx^x'r) 


dxdx'. 


Moreover, for all a; S £1, it holds that 

f Sp{x,x')dx' = j p^'^’'{x,x')dx' — j p^{x,x')dx' = 1 — 1 = 0. 

J E J E J E 

Since the transition probability function is smooth w.r.t. the parameter vector 9, a Taylor 
series expansion to Sp gives. 


Sp{x, x') = e^'ZJgp\x, x') + 0(|ep) . 
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Thus, we finally obtain for all i = 1, that 


p^{x, x') 


dxdx' + 0(|ep) 


= J J i^i_i{x)p^{x,x)Wg\ogp^{x,x')We\ogp^{x,x')'^dxdx'"je + 0{\e\^) 


— + 0{\e\^) 


where, 


IniQi) ( p®(a;,a;')Ve ,log/(a;,a;')Ve logp®(a;,a;')^da;' 

' U E 


is the instantaneous FIM associated to the instantaneous relative entropy. 

Consequently, the pathwise FIM ^(Qq.j.), i.e., the Hessian of the pathwise relative en¬ 
tropy at point 6, is given by 

i{Ql^) =i{v^) +j2MQl) > 

where logz^®(x)Ve logi^®(x)^] is the FIM of the initial distribution. □ 


Remark 1: Let ‘0’ denote the product operator of two distributions (i.e., v'^p{A xB) = 
J^p{x, B)i'(x)dx). Then, the instantaneous relative entropy can be written as a relative 
entropy of the probability measure ® p^ w.r.t. the probability measure Vi_i ®p^'^’^. 
Mathematically, 

n (Q® I = n (vl, ® /1 ® /+') , 

and similarly for the associated instantaneous FIM it holds that 

MQl) . 

Remark 2: The instantaneous relative entropy % [Qf \ is different from and should 

not be confused with the relative entropy of the unperturbed distribution at the i-th (or the 
{i — l)-th) time instant t'f (or w.r.t. the respective perturbed distribution (or 

Indeed, it holds that 


n {Q! I = 7^ {.I, 0 / I .1, 0 /+^) ^ 7^ (.f I , 

as well as H (Qf | ^ B. {yf_i \ Moreover, an explicit formula for the probability 

distribution at the i-th time instant vf is generally not available making the computation of 
the relative entropy TZ {yf \ intractable. On the other hand, the instantaneous relative 
entropy % (Qf | can be computed in a straightforward manner as a statistical average 

since it incorporates only the transition probabilities which are known functions. 

Stationary regime: In the stationary regime, the initial distribution is the stationary 
distribution, Thus, for all i = l,...,r it holds that vf = and the instantaneous 

relative entropy is a constant function of time that is equal to the relative entropy rate. The 
next corollary presents explicit formulas for the relative entropy rate and the associated FIM. 


Corollary 3.1. The relative entropy rate equals to 


p®(x,x') 


n (Q^ I g^+^) = P^ (x, x') log dx' 

Similarly, the FIM associated to the RER is given by 


=E^« 


[ p®(x,x)Ve logp®(x,x')Ve logp®(x, x')'^dx' 

J E 


(13) 


(14) 
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Proof. Bot h fo rmul as a re obtained by substituting the stationary distribution /r® to the place 
of in (10) and (12). □ 


4 Continuous-time Markov chains 


Let be a continuous-time Markov chain with countable state space E. The pa¬ 

rameter dependent transition rates, denoted by c^{x,x'), completely define the continuous¬ 
time Markov chain. The transition rates determine the updates (jumps or sojourn times) 
from a current state a; to a new (random) state x' through the total rate X^{x) = J^x'eE ^') 
which is the intensity of the exponential waiting time for a jump from state x. The transition 
probabilities for the embedded Markov chain {a;n}„>Q defined by Xn ■= where is the 

instance of the n-th jump are p^{x, x') = 

Assume another jump Markov process defined by perturbing the transition 

rates by a small vector e € K^. Moreover assume that the two path probabilities Qp.y and 
are absolutely continuous with respect to each other which is satisfied when c^{x, x') = 0 
if and only if c^^'^{x,x') = 0, \/x,x' G E. The following theorem presents the decomposition 
of the pathwise relative entropy for the case of continuous-time Markov chains. 

Theorem 4.1. (a) The pathwise relative entropy for the above-defined continuous-time 
Markov chain is decomposed as 


^ (Qo:T I QV't) + [ n{Qt\ dt 

Jo 


where the instantaneous relative entropy equals to 

cO{Xt-,Xt) 


n{Ql\Ql+^) = 


X%Xt_)\og 


c«+^(At_,At) 


- (A^(At)-A®+'(Ai)) 


(15) 


(16) 


(b) Under smoothness assumption on the transition rate function c®(-,-) for the parameter 
9, the pathwise FIM is also decomposed as 


X{QI,t) + rin{Qt)dt , (17) 

Jo 

where the instantaneous pathwise FIM is given by 

I«(g®) =EQgjA«(At_)Velogc^X*_,At)Velogc®(At_,At)^] . (18) 

Proof, (a) As in the discrete-time case, the key element is an explicit formula for the Radon- 
Nikodym derivative. The Radon-Nikodym derivative of the path distribution Q^.j- w.r.t. the 
perturbed path distribution Qgjjf has an explicit formula known also as Girsanov formula 

min]. 


dQ^-T 

dQ^oP 


i{Xt}Lo) 


E^+^{Xo) 



c\Xt-,Xt) 

c^+^{Xt_,Xt) 


dNt- 


[A®(At)-A®+^(At)] 



where (reps. 1 /®+*^) is the initial distributions of (resp. {AltjtgR^) while Nt is 

the counting measure, i.e. counts the number of jumps in the process up to time t. Using 
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the Girsanov formula, the pathwise relative entropy is rewritten as 


[QI-.t I Ql%^) 


= E. 




= E 


Qo.i 


log 






E^ 




c%Xt_,Xt) 

log ..^ dNt 


/o ^c^+YXt-,Xt) 




[ [X^{Xt)-X<>+YXt)]dt 

Jo 


Exploiting the fact that the process Mt '■= Nt — Jq X^(Xt)dt is a martingale, we have that 


E^ 


c^(Xt-,XY ,,, 

log —;r^—r;;-^rr dXt 


= Er 


r xYxt-) log 

Jo 


Iq ac9 + e(Xt-, Xt) 

Thus, the pathwise relative entropy is rewritten as 

n {qI,t I qIYY 


c\Xt_,Xt) 

cf>+YXt-,Xt) 


dt 


= 7^ fj/® 1!/®+") + E^ 


^TZiiyYiy' 


e I ,,e+e\ 


«10:T 

rT 


E, 


Qi. 


f X^iXt_)\og 

Jo 

A®(Xt_)log 


c<^+YXt-YXt) 

Y>{Xt,Xt) 


- (X^Xt) - X'^+YXt)) dt 


c<>+YXt-,Xt) 


- (X^iXt) - X<^+YXt)) 


dt 


= 7^ I + rn{QUQt^Y dt 

Jo 


where the instantaneous relative entropy is defined as 

c\Xt,Xt) 


n (Q® I gro = Egg, 


A'^(Xt_)log 


c<^+YXt-,Xt) 


- {X<^{Xt)-X^+YXt)) 


(b) Even though not directly evident from (16), the instantaneous relative entropy for 
continuous-time Markov chains is locally a quadratic function of the parameter vector 0. 
Indeed, defining the rate difference Sc{x,x') = c®+'^(x,x') — c®(x,x'), the instantaneous rel¬ 
ative entropy can be rewritten as 


n (Q® I 

= -E, 


^Qlt 


= -E, 


QL 


xyx,.) log (l + - Eg®, [A^(^*) - A^+^(V)] 

f SciXt-,Xt) _ 1 f 6c{Xt-,Xty 


,, 6c{X,.,X,) 1 fSc{X,.,X,)Y ^ 

A i^^-)[yt^x,_,xY-2[?lX,.:XY) +Oi\Sc{X,_,XY\)^ 

TEgg^^ [A^+^(Xt)-A^(Xt))] 

^Eg®, 


= iE,. 


c^{Xt-,Xt) 


0(|dc|3) 


Under smoothness assumption on the transition rates in a neighborhood of parameter vector 






































9 a Taylor series expansion of Sc{x,x') = e'^'\/gc^{x,x') + 0(|ep) results in 


n {Qt I q: 




2 ^0:t 




0(|<5c|3) 




1 

= -e^E^e 

2 ^0:t 


= ie^I«(g®)e + 0(|e|3) 


Vec^(Xt_,XOVec"(Xt_,XO 


n 


c\Xt-,XtY 


e + 0(|en 


where 

Tn{Qt)=^Ql.^ [>^''{Xt-)Ve\ogc'^{Xt-,Xt)Ve\ogc\Xt_,Xtf] , 

is the instantaneous FIM. Finally, the pathwise FIM is obtained from a straightforward 
expansion of each element of the pathwise relative entropy in terms of e. It is given by 

I{QI,t)=I{u^)+ r Tn{Ql)dt, 

Jo 

where l(^^) is the FIM of the initial distribution while I-h (Qt) is the instantaneous pathwise 
FIM computed above. □ 


Stationary regime: In the stationary regime, the instantaneous relative entropy is a con¬ 
stant function of time since at each time instant the distribution of the states is the -typically 
unknown- stationary distribution. The following corollary presents explicit formulas for the 
relative entropy rate and the associated FIM at the stationary regime. 

Corollary 4.1. The relative entropy rate is equal to the ergodic average 


H (g"" I = V [ E c^a:,a:')log 

x'^E 


c^{x, x') 
c®+'^(a;, x') 


{\\x)-\<^+^{x)) 


(19) 


while the FIM of the relative entropy rate, computed as its Hessian, has explicit formula 
given by 


In{Q^)=E^e 


c^{x, x')Ve logc®(x, x')Vg \ogc^{x, x') 


Lx'^E 


( 20 ) 


Proof. At the stationary regime, the instantaneous relative entropy is rewritten as 


n {Qt I Q 




= E, 


Qt.t 


\\Xt_)\og 


c^{Xt-,Xt) 


= E„ 


c\x,x')\og 


c^+-{Xt-,Xt) 

c^{x, x') 


..6+e 


x' ^E 


{x,x') 


-{\%Xt)-\<^+^{Xt)) 

— (A®(x) — A^^‘^(x))j . 


( 21 ) 


□ 
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5 Stochastic differential equations - Markov processes 

driven by a stochastic differential equation of the 


Consider a Markov process Xt £ 
form 

dXt = b^{Xt)dt + a{Xt)dWt 


( 22 ) 


where b^{-) is the drift function depending on the parameter vector 9, a(-) is the state- 
dependent diffusion matrix, Wt is a d-dimensional Brownian motion while z/® is the initial 
distribution of the process. Let Qg.'p denote the path space distribution for a specific pa¬ 
rameter vector 9. Consider also a perturbation vector, e G and denote by path 

space distribution of the perturbed process, Xt driven by the stochastic differential equation 
(221 with perturbed drift function. 

Under appropriate assumption on the components of the stochastic differential equation, 
the pathwise relative entropy can be decomposed as in the previous cases and an explicit 
formula for the instantaneous relative entropy can be estimated as the following theorem 
asserts. 

Theorem 5.1. (a) Assume that the diffusion matrix, cr(x), is invertible for all x £ and 
Eg. Jexp { r \a-\Xt){b^+^{Xt) - b^Xt))]^}] < oo (Novikov condition) . 

'J 0 

Then, the pathwise relative entropy for the above-defined Markov process is decomposed as 


n {qIt I Qiy) =n{n^\ + n{Ql\ qI+^) dt 


(23) 


where the instantaneous relative entropy is equal to 


n{Qt\Qt^^) = ; e ,. 


(24) 


(b) Assume further that the drift function b^{-) is smooth w.r.t. the parameter vector 9. 
Then, the pathwise FIM has a similar decomposition and the instantaneous FIM is given by 


I'H{Qt)=E,e [[Xeb\xfiaix)aixf)-^Xeb\x)] 


(25) 


where Xgb^f) is a d x K matrix containing all the first-order partial derivatives of the drift 
vector (i.e., the Jacobian matrix). 

Proof, (a) The inversion of the diffusion matrix is a necessary assumption for the well¬ 
poshness of the Novikov condition which in turn suffices for the two path distributions Q^-rp 
and to be absolutely continuous w.r.t. each other |24j. Additionally, the Girsanov 

theorem provides an explicit formula of the Radon-Nikodym derivative [24j which is given 

by 


dQoiT 


dQn. 


,9-\-e 

0:T 


({(^*) }Lo) = ^ (^o) exp I - £ uiX.fdW, - |«(A*) p 


dt 


where u{x) = a ^{x)[b^~^’^{x) — b^{x)). Furthermore, it holds that 


Wt := / u(X,)dt + Wt , 
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is a Brownian motion w.r.t. the unperturbed path distribution Qq.^, meaning that, for any 
measurable function /(•), it holds Ege ^ f{Xt)^dWt] = 0. Then, 


TZ (QoiT 

^0;T 


log 








j-T 1 j-T 

\og^{Xo)-J^ u{XtfdWt--J^ \u{Xt)\^dt 


dv^ 


-E^ 


u{XtfdWt 


+ :^Eq; 


2 ^0:T 


\u{Xt)\^dt 


—E^» 

2 ^0:: 

rT 1 


\u{Xt)\‘^dt 


hence the instantaneous relative entropy is explicitly given by 

n {Ql I = ^E,. [\a-\x){h^+^{x) - b\x)) r 


(b) Due to the smoothness assumption, a Taylor expansion of the drift function around the 
point 9 results in 6®+'^(a:) — b^{x) = Xgb^{x)e + 0{\e\^) where Xgb^{-) is a, d x K matrix 
containing all the first-order partial derivatives of the drift vector function (i.e., the Jacobian 
matrix). Then, it is straightforward to obtain from ( p4| ) to get 

In{Qt)=E,e[Xgb<^{xf{a{xMx)^)-^Xgb^{x)] . 

□ 


Stationary regime: In the stationary regime, the instantaneous relative entropy becomes 
a constant function of time since the distribution of the process equals to the stationary 
distribution denoted by /i® for all times. The following corollary presents explicit formulas 
for the relative entropy rate and the associated FIM. 


Corollary 5.1. Under the same assumption of Theorem 5.1 and let Xq ^ pP. Then, the 
relative entropy rate equals to 


(Q® I 



cr ^{x)(b^'^'^{x)— b^{x))\'^ 


(26) 


while the FIM associated with the relative entropy rate is given by 

Iu{Q^) = ¥.^e[Xgh\xf{a{x)a{xf)-^Xgb\x)] . (27) 


We finally remark that a popular method for modeling non-equilibrium systems in atom¬ 
istic and mesoscopic scales is based on the Langevin equation. Langevin equation is a 
degenerate system of stochastic differential equations whose sensitivity analysis based on the 
relative entropy rate and the associated pathwise FIM was performed in |25) . 


6 Demonstration example 


In this section we give a numerical example of the pathwise relative entropy § and 
pathwise FIM ® for a continuous time Markov chain model. More specific, a biological 


reaction network is considered and the quantities instantaneous RE (161 and instantaneous 


FIM (181 are presented as a function of time. 
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6.1 Continuous time Markov chains: an EGFR model 

In [2S] , Kholodenko et al. proposed a reaction network that describes signaling phenomena 
of mammalian cells Eaumn]. The reaction network consists of IV = 23 species and M = 47 
reactions. The propensity function for the Rj reaction, j = 1,..., 47 and j ^ 7,14, 29, obeys 
the law of mass action [2], 

= (-.)(--). ( 28 ) 

k 

for a reaction of the general form + l3jBj —^ where Aj and Bj are the reactant 

species, aj and Pj are the respective number of molecules needed for the reaction, kj the 
reaction constant and and xb^. is the total number of species Aj and Bj, respectively. 
The binomial coefficient is defined by (^) = ■ The propensity functions for reactions 

i? 7 , i?i 4 , i ?29 are being described by the Michaelis-Menten kinetics, see [5], 

a, (x) = / {K, +^A,), j = 7,14, 29 , (29) 

where Vj represents the maximum rate achieved by the system at maximum (saturating) 
substrate concentrations while Kj is the substrate concentration at which the reaction rate 
is half the maximum value. The parameter vector contains all the reaction constants, 

0 [ki, . . . , A)g, fcg, . . . , A)i 3 , A)i 5 , . . . , ^287 ^30) ■ ■ • 7 k„, 1 / 7 , Kj, Vi 4 , Ku, V 29 , K29f . (30) 

In this study the values of the reaction constants are the same as in [26] . 

For the initial data and parameters chosen in this study, the time series can be split 
into two regimes: (a) a transient regime that approximately corresponds to the time interval 
[0,50] and (b) a stationary regime which approximately corresponds to the time interval 
[50, 00 ). 

Next, we discuss two sensitivity measures of the process {xVs}‘^q: the instantaneous RE 
defined in ( [T^ 

/ ({x.}l=o) =^(QH Qf+O 



with t € [0, T] and the averaged RE, defined as 


9 {{Xs}Uo) =] Q^^) ds, t G [0, T] . (32) 

In Figure Q the two sensitivity measures are presented for T = 100. As expected, the 
averaged RE is smoother than the instantaneous RE while some of the qualitative charac¬ 
teristics remain. On the other hand, quantitative characteristics, such as the time that two 
instantaneous RE are crossed, are not preserved in the averaged RE. The averaged RE in the 
interval [0, t] should be interpreted as a measure of the information accumulated in the the 
whole interval while the instantaneous RE is a measure of the information at the time instant 
t. Moreover, the observable can used as part of an upper bound for a different sensitivity 
measure, see m for a detailed discussion. 

Let us define a different sensitivity measure as the relative difference between the k-th 
species of two systems were the £-th parameter of the second is perturbed by e'. 


■ — 


-y6 


(33) 
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Figure 1: The instantaneous RE, defined in (15), for the EGER model (left) and the averaged 
RE, defined in (32) (right). 


where is a vector with zeros everywhere and e' in the Gth position. In the following 
examples the value of e' for a perturbation in the £-th parameter is e' = O.10f. By summing 
over all species we obtain a total sensitivity measure, which is only indexed by the parameter 
index and time, 

1 ^ 

= (34) 


By observing the first row of Figure]^ which shows the instantaneous RE (161 for £ = 10 
and ^ = 29, we learn that perturbations in the 10-th parameter have large sensitivity for small 
times and as time varies the sensitivity is getting smaller. On the other hand, perturbations 
in the 29-th parameter have small influence on the system for small times while for larger 
times the sensitivity becomes significant. These observations are in good agreement with the 
sensitivity measure (35) presented in the second and third row of Figure]^ 

Moreover there is a crossing in the instantaneous RE which happens around t = 42. After 
this time the sensitivity in parameter £ = 29 becomes more significant than the sensitivity in 
parameter ^ = 10. This is again in agreement with the behavior of the sensitivity measure 
(351. This observation shows that the transient regime is more sensitive in perturbations in 
the 10-th parameters while the equilibrium is more sensitive in perturbations in the 29-th 
parameter. 


7 Conclusions 

In this paper we presented two pathwise sensitivity measures for the analysis of stochastic 
systems; the instantaneous relative entropy and its approximation, the instantaneous Fisher 
information matrix. These sensitivity tools serve as an extension to transient processes of the 
sensitivity tools presented in m- Three examples, discrete-time Markov chains, continuous¬ 
time Markov Chains and stochastic differential equations, were presented as an application 
of the new sensitivity measures. In Section 6 we demonstrated, in a biological reaction 
network, how the proposed pathwise sensitivity measure can be applied to transient, as well 
as in steady state regimes. 

Finally, the pathwise sensitivity method is directly connected to a different sensitivity 


13 





































time 


Figure 2: Instantaneous RE (15) for the EGFR model and parameters 10 and 29 (first row). The 
total sensitivity of the system, as defined in (34), due to perturbations in the 10-th parameter 
(second row) and due to the 29-th parameter (third row). 


measure that depends on specific observables. More specifically, if we define the sensitivity 
index (SI) of the Tth observable to the A:-th parameter as 


d 


Sk,, = [/, ({XjLo)] . 


then IFIM (18) serves as un upper bound of Sk^e through the inequality. 


< ^VarQg^(/,)^X(^ 


T)k,k ’ 


(35) 


(36) 


where F = (/i,..., Jl) is a vector of observable functions. This inequality follows by rearrang¬ 
ing the generalized Cramer-Rao bound for a biased estimator EHIEI]- Due to low variance 
of the estimator of IFIM compared to the variance of a finite difference estimator of Sk,e, 
the estimation of the right hand side of (36) is faster that that of the left hand side of (36). 
In [15| the authors use this inequality to efhciently screen out and exclude low sensitivity 
indices under a pre-specified value and then perform a coupling finite difference algorithm 
[5] to accurately estimate the remaining sensitivity indices Sk,e- In Figure]^ the estimated 
Sis for the EGFR model discussed in Section are ordered in the parameter direction using 
only the IFIM (18). Notice that the Sis are then grouped into four distinct regions. For a 


detailed presentation of this methodology we refer to m- 
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